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Abstract. The aim of this research was 

to examine the effects of the Augmented 
Reality (AR)-supported teaching activities 
on the academic success, motivation of 
students towards science learning and at- 
titudes of students towards AR application, 
in the scope of the seventh grade science 
course, “Solar System and Beyond” unit. For 
this purpose, “Solomon Four Group Model’, 
which controls the internal and external 
validity, was used. Research was conducted 
with 120 students in two different schools 
and involved two experimental and two 
control groups established by random 
method. Experimental (1) and Control (1) 
groups received data collection tools as 
pre-test and post-test, while Experimental 
(2) and Control (2) groups received only 
post-test. During the six-week research, the 
students were taught in the experimental 
groups using the ‘Star-Tracker, Spacecraft, 
iSolarsystem, Space-4D’ mobile AR ap- 
plications, while the control groups were 
taught with the activities envisaged in the 
curriculum. “Solar System and Beyond Suc- 
cess Test” and the “Students’ Motivation to 
Science Learning” scale, were used as data 
collection tools. Findings from the research 
showed that teaching with AR applications 
significantly alters the success of students 
and motivation towards science learning. 
This showed that teaching with AR applica- 
tions is effective. 
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Melek Sentiirk Basgill 


Introduction 


The rapid development of technology enables digital tools and applica- 
tions to become an ordinary part of our social as well as learning experiences 
by allowing them to integrate more and more into daily lives. Augmented 
Reality (AR), which arises from these technological developments, is a technol- 
ogy in which real and virtual objects are allowed to interact simultaneously 
and virtual-world objects are superimposed on real images (Azuma, 1997). 
In other words, through these applications, an object that is not actually 
there is perceived as being there through the display of mobile devices. With 
these developments in mobile devices, the AR technology is used in many 
sectors such as health, marketing, education, cinema, advertising and design. 

In engineering and architecture education, it is very important to use AR 
applications since students need to animate objects in their minds in three 
dimensions. Quintero, Salinas, Gonzalez-Mendivil and Ramirez (2015) de- 
signed an innovative and educational AR application aimed at strengthening 
mathematical cognitive skills for engineering students. With this application, 
it was determined that mathematical objects were learned more effectively 
than teacher drawings or verbal descriptions and students did not have dif- 
ficulties to imagine in three dimensions. 

In the teaching of mathematics, the usage of AR in the classroom is 
quite important since teaching materials such as papers, pencils, ruler are not 
sufficient to visualize geometric shapes three dimensionally in the minds of 
students. In a master thesis study, Glin (2014) carried out a semi-experimental 
study with 188 sixth grade students with the aim of investigating the effect 
of supplementing the mathematics course with AR applications named 
BuildAR on the students’ spatial visualization abilities and academic success. 
Three-dimensional shapes of rigid bodies were created with BuildAR during 
application. As a result of the study, the students found the AR applications 
fun and remarkable. 

Technology enhanced teaching equipment aids students in visualizing 
physical concepts and understanding the real world situations and problems. 
Ozarslan (2013) conducted a semi-experimental study with 63 third-grade 
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computer science students in order to determine the effect of using AR-enriched learning material on the success 
and satisfaction levels of students. A functionally enriched application called OptikAR was developed. Eventually, 
it was determined that the application positively affected student success and satisfaction levels. 

In biology education, the usage of interactive AR applications in the teaching of many topics such as the di- 
gestive system, circulatory system, plant diversity, insect diversity and classification, ecosystem, protein synthesis 
resulted in more permanent and effective learning experiences. Yusoff, Zaman and Ahmad (2011) have worked 
with 63 undergraduate students to determine their perception and acceptance of the AR technology. In the pro- 
cess, the structure of the skin has been observed in three dimensions. As a result of the study, they concluded that 
students would like to use this technology in the future. According to the result of another research, the students 
have agreed that the AR platform was a new and useful tool that could be successfully applied for learning purposes 
in the educational process (Lamanauskas & Bilbokaité, 2009). 

It is also possible to observe the atoms and molecules in three dimensions via AR technology. Nunez, Quiros, 
Nunez, Carda and Camahort (2008) has carried out a study with 15 college students on inorganic chemistry using 
AR applications. As a result of the study, it was seen that the three-dimensional crystal structures developed the 
students’ spatial abilities and became an effective support tool that increased their success. 


Significance and Aim of Research 


It has become easier to bring AR applications into the class and laboratory as teaching tools using mobile 
devices. In this way, mobile AR applications can be used as an educational tool for teaching of space and universe 
learning as well as all other science fields. Observing the solar system and the planets clearly cannot be possible 
in traditional classroom environment. For this reason, AR applications are seen as effective tools in providing ac- 
cess to and visualization of the concepts in astronomy which are difficult to learn. Thus, the unknown features and 
inaccessible locations of the universe are discovered by the students. They are provided with an opportunity to 
experience and learn the universe in a realistic and better way. Sahin (2017) has designed supporting material and 
developed AR activities for the “Solar System and Beyond" unit by consulting with educational design experts with 
the aim of investigating the effects of supplementing science instruction with AR-enhanced teaching materials on 
the success levels and attitudes of secondary school students. As a result of this semi-experimental study, it was seen 
that there was a significant difference in the success levels and attitudes towards the course between the students 
who took the course in a learning environment supported by AR technology and the success levels of students 
who were taught in the traditional way. It was also determined that the AR application had a positive influence on 
the attitudes of the students. In order to investigate the effect of the use of AR applications in the Solar System and 
Beyond unit on the academic success of students, Kirikkaya and Senttirk (2018) conducted a quasi-experimental 
and pre-test & post-test controlled group study with 45 seventh grade students. As a result of the study, it was 
determined that the AR applications applied in the experimental group increased the academic success. 

Since the idea that AR can be used in educational environments has recently become widespread, many 
applications have been designed to be used in science education. These include Spacecraft-3D, Star Tracker, Star 
Chart, and Spacecraft -4D, Elements 4D, Dinosaur, Quiver-3D, Octaland-3D, Animal-4D, Augmenter, ARMolVis, Rapp 
Chemistry, Animal Cell, Soace Adventure, iSolarsystem and Solarsystem-AR applications. 

It is known that the students of primary school age learn more easily about the objects they are able to see 
and touch. It is especially important to be able to visualize objects such as the solar system, planets, telescopes 
and spacecraft in order to comprehend the concepts in “Space and Universe” learning domain. AR technology is 
a good alternative in terms of three-dimensional visualization of these concepts. The fact that the AR technology 
provides interactive and reality-enhancing environments suggests that such practices can be used in teaching 
subjects related to the world and the universe. Considering the fact that the usage of this technology is yet in 
early stages and not yet widely used in classroom activities and the limited number of reports assessing its use in 
science education in our country, it has been considered as worthwhile research topic. Given the limited number 
of studies in which the Solomon four-group model was used, this research also filled a gap in the subject and the 
AR technology as well as the method employed. In addition, while the researches for AR applications have only 
just begun, the realization of a fully experimental work based on the Solomon four-group model which protects 
both internal and external validity reveals the originality of the research. Through this perspective, it is aimed to 
research the effect of teaching with AR-enhanced on the academic success and motivation of students in the context 
of “Solar System and Beyond” unit in the Science course. In this respect, the research questions were as follows: 
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1. Is there a significant difference between students in the experimental and control groups in terms 
of academic success and motivation upon application of the experimental procedure and pre-test 
stimulation? 

2. Does the experimental procedure lead to a significant difference between academic success and mo- 
tivation pre-test and post-test scores of the groups stimulated by the pre-test in the science course? 

3. When pre-test is controlled, does the experimental procedure make a significant difference between 
the academic success and motivation post-test scores of the students in the pre-test groups? 


Research Methodology 
Research Design 


The use of experimental designs is recommended for strengthening scientific validity in educational research. 
Experimental design is a research area where data are generated to explore the cause-and-effect relations between 
variables under the control of researcher (Karasar, 2006). It is very important to ensure internal and external valid- 
ity in this research area. By ensuring internal validity, it is determined whether the independent variable causes 
the change in the dependent variable, whereas ensuring external validity allows the researcher to generalize the 
results of the study to individuals in the population. This research conducted by using “Solomon Four-Groups 
Design” which is one of the most powerful models that ensures internal and external validity at the same time 
(Fraenkel & Wallen, 2009). This way, this research enabled the checking of the presence of pre-test sensitivity and 
thereby added a higher external validity in addition to internal validity. In this context, a symbolic representation 
of the Solomon four-group model was shown in Table 1. 

As shown in Table 1, research groups included two experimental and two control groups. Pre-test scales were 
applied to one of the groups whereas no pre-test was applied to the other experimental and control groups. In 
other words, in addition to the effect of the independent variable on the result through the pre-test and post-test 
applied to the first experimental and control groups, it was determined whether the pre-test has any effect on the 
results by not applying the pre-test to the second experimental and control group (Rosnow & Rosenthal, 2002). 


Table 1. Asymbolic representation of the Solomon four-group model. 


Groups Impartiality Pre-Test Treatment Post-Test 
Experimental (1) R 0, X 0, 
Control (1) R O, 0, 
Experimental (2) R X 0, 


Control (2) R 0 


O, Experimental (1) pre-test; O,-Experimental (1) post-test; O,:Control (1) pre-test; O,;Control (1) post-test; O,:Experimental (2) post- 
test; O,;Control (2) post-test; X:Experimental intervention, R:Neutrality in the formation of groups 


In experimental designs, multiple groups were used; usually experimental and control groups, and these 
groups should be generated via unbiased assignment (Creswell, 2003). For this reason, research groups were 
formed through random assignment in this research, so that any changes that could be observed in terms of 
results between the experimental and control groups are unbiased. These steps were performed according to the 
Solomon four-group model given in Figure 1. 

As shown in Figure 1, experimental groups were educated with the help of AR applications appropriate for the 
daily plans prepared by researchers, whereas control groups were instructed simply according to the daily plans. 
“Solar System and Beyond Success Test” and “ Students’ Motivation to Science Learning “ scales were applied as 
pre-test to the Experimental (1) and Control (1) groups prior to start of the study and as post-test to all groups at 
the end of the research. 
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SOLOMON FOUR GROUP DESIGN DIAGRAM FOR THIS RESEARCH 
SCHOOL A |- “| SCHOOL B 


Control Group 1 
(N= 30) 








Control Group 2 
(N= 30) 


Experimental Experimental 





Group | (N= 30) Group 2 (N= 30) 
















Pre-Test Application 

*Solar System and Beyond 
Success Test (SSBST) 
*Student Motivation to 
Learn Science" scale 
(SMSL) 


Pre-Test Application 
*Solar System and Beyond 
Success Test (SSBST) 

*Student Motivation to 
Learn Science" scale 
(SMSL) 


Activities in the curriculum The curriculum by activities 
supported with AG. 








Post-Test Application 
* Solar System and Beyond Success Test (SSBST) 
* Student Motivation to Learn Science" scale (SMSL) 





Figure 1. The experimental design. 


Research Groups 


Before the experimental process, the groups to be included in the research were selected in two state schools 
in the district of Dilovasi, Kocaeli province in Turkey in the second semester of 2016-2017 education years. In the 
selection attending groups, one-way analysis of variance (ANOVA) was done to determine whether there was a 
significant difference between the year-end averages of science lecture of all seventh grades in these two schools 
(Table 2). 


Table 2. Results of One-way ANOVA for the science lecture report card averages of the seventh- grade students 
in school A and B during the academic year of 2016. 


Source of Variance Sum of Square df Mean of Square F p 
Between Groups 3237.27 8 404.66 1.53 5° 
Within Groups 72452.61 274 264.43 
Total 79073.66 282 
*p>.05 
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As seen in Table 2, there was no significant difference between the average grade of Science lectures in the 
2016 academic year of all seventh grade groups in two schools (F., ,.=1.53; p>.05). Since there was no statistically 
significant difference between the groups, four randomly selected groups were included in the research. Experi- 
mental and Control groups were assigned unbiasedly from these four groups. As a result, research was carried out 


with 120 students which were grouped as experimental and control groups (Table 3). 


Table 3. Demographic characteristics of students in the research group. 





N 
School Groups 
Young Woman Young Man Total 

Control (1) 17 13 30 

School A 
Control (2) 15 15 30 
Experimental (1) 19 11 30 

School B 
Experimental (2) 13 17 30 


Due to the lengthy research period, the researcher's own activities and the need for a technical infrastructure, 
the schools involved in the research were determined by the easy-to-reach sampling method. The reason for choos- 
ing two different schools in the research was to ensure that the control group is not affected by the experimental 
procedures. In this context, it was made a point of choosing the experimental groups from the same school. In ad- 
dition, in the classrooms except the groups participating in the research in the same school where the experimental 
procedures were carried out, the lessons were also taught with the support from the AR applications in order to 
avoid the feeling of exclusion and to benefit from the same activities. The researcher has done the activities in all 
classes in the school where the research was conducted. 


Instrument and Experimental Process 


Solar system and beyond success test (SSBST): The success test, which is formed of 20 questions, was developed 
by Kirikkaya and Sentiirk (2018) in order to measure the success of students for the subjects of “Solar System and 
Beyond” unit in the seventh grade. The test, with a confidence of .83, was evaluated over 100 points and details 
are as follows: the total number of correct questions was multiplied by five, where one point was given for each 
correct question and zero points for each wrong question. In this study, pre-test reliability coefficient for “Solar 
System and Beyond Success Test” was .70; and the post-test reliability coefficient was .77. 

Students’ motivation to science learning scale: In order to determine the motivation of primary school students 
for science learning, “Students’ Motivation to Science Learning (SMSL)” scale, which was used in the study, was 
developed by Tuan and colleagues in 2005. The Turkish adaptation study was conducted by Yilmaz and Huyuguzel 
Cavas (2007), and the reliability of the 5-point Likert type SMSL scale was found to be .87, and it was determined 
that the reliability level for the measurement tools that can be used in the researches was provided. In this research, 
reliability coefficient of pre-test was calculated as .82 for the SLMS scale and the final test was calculated as .89. 

The experimental process was structured by the researchers with preparing lesson plans and activities. These 
lesson plans and activities were organized in accordance with the AR applications and in line with the achievements 
of the Science Education Program. The lesson plans prepared for the control groups were organized according to 
the activities envisaged in the science curriculum. In addition, in order to provide the balance between the research 
groups, it was found appropriate for the researcher to lecture the courses in both experimental and control groups. 
The basic steps followed in the experimental process are shown in Figure 2. 

According to Figure 2, SSBST and SMSL Scale was applied as pre-test before the experimental process and AR 
technology was introduced to the experimental groups. It was necessary to inform students beforehand since they 
had not already met with AR applications and that these applications required technological skills. The research 
was conducted for a total of six weeks including four weeks experimental application, one week pre-test and one 
week post-test applications. In this process, activities including AR applications were carried out by researchers in 
the experimental groups. In the control groups, activities included in the curriculum were carried out (Figure 2). 
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Tim i ions. 
e Realised Interventions Groups 


Determination of research groups. 


Solar System and Beyond Success Test. (SSBST) Experimental (1) 
Students' Motivation Scale for Science Teaching Control (1) 
(SMSL)Pre-test application 


Introducing experimental process process (AG Experimental (1) 
applications) to students. Experimental (2) 


Performing activities supported by AG 
applications in the Course of Science (performing 
the experimental process). 

Celestial bodies = Space Adventure, Star Chart, : 
Star Tracker, Nebulae, Experimental (2) 
Solar System siSolarsystem, Space 4D+, Solar 

System 3D 

Space Researches Spacecraft 4D, iSolarsystem, 


2nd-5th Weeki Astronaut 
Processing of the course by the methods and 


techniques foreseen in the Science Education 
Program. 












Ist. week 


Experimental (1) 





Control (1) 
Control (2) 






Solar System and Beyond Success Test. (SSBST) 
6th Week Students' Motivation Scale for Science Teaching All Groups 
(SMSL) 


Figure 2. Procedures followed during experimental process. 


“Sky Bodies” subject was taught with Space Adventure and Star Tracker AR applications in experimental groups. 
Students have been able to observe meteorite, meteor, meteor crater, star clusters (Ursa Minor, Ursa Major, and 
dragon), Orion nebula, Andromeda galaxy, Milky Way galaxy in 3D observation. Moreover, owing to this applica- 
tion, students had an opportunity to observe that the sun was a star and turned around its own axis and the stars 
were not as the one on the Turkish flag, but were spherical in shape. 

The “Solar System” subject was taught with iSolarsystem and Space 4D AR applications in the experimental 
groups. One of the best achievements of these AR applications was that they showed very well that the planets 
are turning around the sun in a certain orbit. Instead of learning the features of the planets from the books in 
two-dimension, students have learned many of their features from the very beginning, such as proximity to the 
sun, satellite numbers, magnitude, rotation speeds around orbits, daily temperature differences and number of 
days to complete one revolution around the Sun. Moreover, students who studied the rotations of the Sun, the 
Earth and the Moon with the application of “iSolarsystem” AR, could better perceive the concepts of time such as 
a day, a year, a month. 

“Space Research subject was taught with Spacecraft 4D and iSolarsystem AR applications in the experimental 
groups. Through these applications, students were able to observe the structure of the satellites in three dimen- 
sions via telescopes. During the experimental process, the students stated that they had heard the name of the 
Hubble Space Telescope. They added that they were observing the sky realistically for the first time, and they felt 
like astronauts and wanted to be astronauts. After four weeks of experimental process, “SSBST” and “SMSL’ scales 
were applied to all groups as the post-test. 
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Data Analysis 


Cronbach’s alpha coefficient was used for the calculating the reliability of the scales. For the determination of 
homogeneity and normality for data of obtaining from scales Levene Test, Shapiro Wilk, “Skewness and Kurtosis” 
were used. For the determination of the independent and combined effect of pre-test and experimental process, 
“Two-Way ANOVA for Independent Measurements’; for the interaction of pre-test and post-test in pre-test applied 
groups, “Two-way ANOVA for mixed measurements (ANOVA) and one-way analysis of covariance (ANCOVA) were 
used as statistical procedures. In addition, “Tukey Test” was used to determine the groups in which the significant 
difference was obtained, and the“Eta Squared” value was used to determine the effect size of statistical significance. 

The data collected in the research were analyzed through SPSS 21 package program. 


Research Results 
In the analysis of the data obtained from the research, it was first examined whether the data could meet the 
basic assumptions of the parametric tests in order to decide which ones to be used. For this purpose, the Levene 


test was performed to determine whether the variances of the data were homogeneous (Table 4). 


Table 4. Homogeneity test results for pre-test and post-test of experimental and control groups. 





Data collection tool Test Levene Test df df2 p 
Pre-Test 3.50 1 58 07 

SSBST 
Post-Test 2.31 3 116 .08 
Pre-Test 04 1 58 84 

SMSL 
Post-Test 51 3 116 67 


In Table 4, it is seen that the variance of the pre-test and post-test success scores of the experimental and 
control groups showed a homogeneous distribution (Pre-test, F=3.50, p>.05; Post-test, F=2.31; p>.05). Likewise, 
the variance of pre-test and post-test motivation scores of the experimental and control groups also showed a 
homogeneous distribution (Pre-test, F=.04; p>.05; Post-test, F=.51; p>.05). It was determined that the data showed 
a homogeneous distribution in terms of success and motivation variables via variance analysis. Then normality 
analysis, which is another assumption of parametric tests, was done. In this research, the normality analysis of the 
data was examined by considering the Skewness and Kurtosis values. According to Simsek (2007), Skewness and 
Kurtosis values should also be considered while testing the assumption of normality. Tabachnick and Fidell (2013) 
indicated that the values of Skewness and Kurtosis should be within the range of +1.5 to -1.5, for admissibility 
limits. The values of Skewness and Kurtosis were given in Table 5 for pre-test and post-test measurement scores 
of this research. 


Table 5. Values of Skewness and Kurtosis for pre-tests and post-tests of experimental and control groups. 





8 Groups Skewness Kurtosis 8 Groups Skewness Kurtosis 
7 Experimental (1) -.12 -.10 B Experimental (1) -,54 -,10 
2 2 
& Control (1) 36 -.79 & Control (1) 41.11 1.46 
R Experimental (1) -.53 -.44 2 Experiment (1) -.33 -.59 
(Jp) ~ — 
Oo) 8 Control (1) -.38 “72 °) 8 Control (1) “51 “17 
8 Experimental (2) 15 -.94 é Experimental (2) -54 1.09 
Control (2) -.27 -.99 Control (2) -.99 1.41 

















Table 5 shows the values of Skewness and Kurtosis for the points that the research groups have taken from 
the data collection tools. Considering the values determined by Tabachnick and Fidell (2013), it was decided that 
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parametric tests should be used, because no dataset was found to exceed the range of +1.5/-1.5 and the data 
showed normal distribution. 


Results Related to the First Question 


The first question of the study was “Is there a significant difference between students in the experimental 
and control groups in terms of academic success and motivation upon application of the experimental procedure 
and pre-test stimulation?” That is to say, this question was related to examining the independent and combined 
effects of the experimental procedure and pre-test. For this purpose, “Two-way Analysis of Variance (2x2 ANOVA)” 
was conducted for independent measurements. Firstly, the arithmetic average and standard deviation of the SSBST 
post-test scores of the research groups were examined (Table 6). 


Table 6. Arithmetic mean and standard deviation values for the SSBST post-test scores of experimental and 
control groups. 


Data collection tool Group N xX SD 
Experimental (1) 30 84.33 12.23 
Control (1) 30 70.07 19.77 
SSBST 
Experimental (2) 30 81.33 13.19 
Control (2) 30 69.67 16.66 


As seen in Table 6, the average SSBST of Experimental group (1) after the application was X oct test 04-33 and 
Control group (1) Xosttest= / 0.07; Experimental group (2) X posttest 8 1.33 and Control group (2) X vost test 09-67. Two- 
way ANOVA was then conducted for the SSBST post-tests of the research groups (Table 7). 

Table 7. Results of two-way analysis of variance for independent measurements for SSBST post-test scores 
of experimental and control groups. 


Source of Variance Sum of Square df Mean of Square F p n? 
Pre-test 86.70 1 86.70 35 56* .00 
Experimental Process 5044.03 1 5044.03 20.34 .0001 15 
Pre-test * Experimental Process 50.70 1 50.70 20 .65* .00 
Total 34930.73 119 
*9<.05 


Without considering the experimental procedure, according to Table 7, there was no significant difference 
between the groups who received and who did not receive the pre-test (F., ,,)=.35; p>.05). That is to say, the pre- 
test stimulation of the students in the Experimental (1) and Control (1) groups before the experimental process, 
did not lead to a significant difference in the success (SSBST) of the students in the research group. When the ef- 
fect of the experimental process was examined, it was found that there was a significant difference between the 
experimental group students and control group students’ academic success (SSBST) post-test scores (F., ,,5=20.34; 
p<.05). In addition to the statistical significance in the difference between the groups, it could be said that the 
experimental process reflected a wide range of effects (n’?=.15). Tukey test was performed to understand between 
which groups that significant difference was. According to this test, SSBST post-test average of the Experimental 
group! (X=84.33) was significantly higher than the Control1 (X=70.07) and Control2 groups (X=69.67) (p<.05). 
SSBST post-test average of the Experimental group2 (X=81.33) was significantly higher than the Control1 (X=70.07) 
and Control2 (X=69.67) groups (p<.05). According to this finding; it could be said that each Experimental group 
differs from the control group in terms of SSBST post-test averages. There was no difference between Experimental1 
(X=84.33) and Experimental2 (X=81.33) groups in terms of SSBST post-test scores, as Control group] (X=70.07) and 
Control group2 (X=69.67) (p>.05). These results showed that teaching the course with AR applications in experi- 
mental groups was effective in increasing the success of the students. Moreover, it was found that the experimental 
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procedure and pre-test had no significant combined effect on students’ academic success (F., ,,,=.20; p>.05). After 


conducting two-way analysis of variance for the success test, the arithmetic average and standard deviation values 
of the post-test scores of the science learning motivation of research groups were examined (Table 8). 


Table 8. Arithmetic mean and standard deviation values of post-test SMSL scores of the experimental and 
control groups. 


Data collection tool Group N X SD 
Experimental (1) 30 136.30 13.94 
Control (1) 30 122.77 16.53 
SMSL 
Experimental (2) 30 131.47 15.35 
Control (2) 30 125.87 17.48 


As seen in Table 8, SMSL for the Experimental group (1) was X rosttest= | 36.30; Control group (1) was 
Xvocetest=! 22.77; the Experimental group (2) was Xpost-test=131.47 and Control group (2) was p enter 125.87. Two- 


way analysis of variance was then performed for motivation post-tests of research groups (Table 9). 


Table9. _—_ Results of two-way analysis of variance for independent measurements of post-test SMSL scores of 
the experimental and control groups. 


Source Sum of Square df Mean of Square F p n? 
Pre-Test 22.53 1 22.53 .09 it .00 
Experimental procedure 2745.63 1 2745.63 10.89 .0001 09 
Pre-test *Experimental Intervention 472.03 1 472.03 1.87 AT* 02 
Total 32486.79 119 


*p>.05 


According to Table 9, there was no significant difference between the SMSL averages of the groups which 
received the pre-test and did not receive the pre-test (F., ,,,=.09; p>.05). That is to say that pre-test stimulation of 
the students before the experimental intervention did not lead to a significant difference in the motivation of the 
students in the experimental (1) and control (1) groups. When the effect of the experimental process was examined, 
it was found that the experimental group and the control group students had a meaningful difference between their 
post-test scores of SMSL and this difference reflected moderate effect Fane=! 0.89; p<.05; n?=.09). Tukey test was 
performed to see between which groups there were significant differences. Accordingly, the motivation post-test 
average of the Experimental group (1) (X = 136.30) was significantly higher than the Control (1) (X=122.77) and 
Control (2) (X=125.87) groups (p<0.05). The SMSL scale post-test average of the Experimental group (2) (K=131.47) 
was significantly higher than the Control group (1) (X=122.77) (p<0.05). According to this finding, it can be said 
that each experimental group differs from the control groups in terms of motivation post-test averages. In addition, 
there was no difference between SMSL scale post-test averages of Experimental (1) (XK=136.30) and Experimental 
(2) (X=131.47) groups. Nonetheless, there was no difference between SMSL scale post-test averages of Control 
(1) (X=122.77) and Control (2) (K=125.87) groups (p>.05). These results showed that teaching the course with AR 
applications in experimental groups was effective in increasing the motivation of the students. Moreover, it was 
found that the experimental procedure and the pre-test had no significant common effect on students’ motiva- 
tion (F...=1.87; p>.05). 


(1,116) 


Results Related to the Second Question 


The second question of the research was declared as “Does the experimental procedure lead to a significant 
difference between academic success and motivation pre-test and post-test scores of the groups stimulated 
by the pre-test in the science course?” That is to say that, this question was intended to determine whether the 
experimental process is effective on groups received pre-test. For this purpose, “Two-way Analysis of Variance for 


370 


URS https://doi.org/10.33225/jbse/19.18.362 


Journal of Baltic Science Education, Vol. 18, No. 3, 2019 


ISSN 1648-3898 /Print/ THE EFFECT OF THE USE OF AUGMENTED REALITY APPLICATIONS ON THE AGADEMIG 
SUCCESS AND MOTIVATION OF 7TH GRADE STUDENTS 
ISSN 2538-7 138 /onines (P. 362-378) 


mixed measurements (2x2 ANOVA)” was carried out to determine whether the scores of the experimental (1) and 
control (1) groups received from the data collection tools significantly differed before and after the experimental 
procedure. Firstly, for the SSBST, the arithmetic mean and standard deviation values for the pre-test, post-test, and 
difference scores of experimental (1) and control (1) groups were examined. 


Table 10. Thearithmetic mean and standard deviation values for the SSBST pre-test, post-test, and difference 
scores of experiment1 and control1 groups. 


Pre-Test Post-Test Difference Score 
Data collection 
tool Group N 7 7 — 
X SD X SD X SD 
SSBST Experimental (1) 30 46.17 16.38 84.33 12.23 38.16 13.16 
Control (1) 30 41.83 20.41 70.07 19.77 28.24 14.45 


As shown in Table 10, before the application, the SSBST score of Experimental group (1) was X rete =46.1 7;and 
after the application, it was A cigs O48, Before experimental process, the academic success score of the Control 
group (1) was Xvretest=41 .83; while the post-,.., score was X ,ost=/ 0.07, after the application. In addition, the differ- 
ence between the post-test score and the pre-test score of the Experimental group (1) was X Xx =38.16; 


post-test pre-test 


while in the Control group (1), this difference was X -X =28.24. Two-way analysis of variance for mixed 


post-test pre-test 


measurements for academic success was then performed (Table 11). 


Table 11. Results of two-way analysis of variance for mixed measurements for SSBST pre-test and post-test 
scores of experimental (1) and control (1) groups. 


Source of Variance Sum of Square df Mean of Square F p n2 
Between-Subjects Factors 28668.80 59 
Experimental procedure 2594.70 1 2594.70 5.77 .02* .09 
Within-Subjects Factors 43262.00 60 
Difference between pre-test and post-test 33067.20 1 33067.20 202.85 .0001* 18 


Difference between pre-test and post-test 
“Experimental procedure 


Total 71930.80 119 
*p<,05 


740.03 1 740.03 4.54 .04* 07 


According to Table 11, it was seen that the SSBST scores of the experimental groups that had lessons with AR 
applications and the control groups that had only lessons with the activities in the program showed a significant 
difference after the experimental intervention, when compared to the start of the experiment; that is to say that 
the increase of the score of the experimental group was significantly higher than the control group (F,, ..=4.54; 
p<.05). Considering that the difference scores were in favor of the experimental group (1) (Experimental group (1), 
X.osetest” Xpre test = 38:16; Control Group (1), X, ose rest Xpre-test= 29:24); it could be said that the support of the program's 
activities with AR applications had a significant effect on the students’ success on the “Solar System” subject and this 
effect was moderate (n?=.07). Since the aim of this study was to test the effectiveness of AR applications in increas- 
ing students’ success, the combined effect of measurement and experimental process factors was emphasized. 
In addition, the key factor tests of measurement and the experimental process were also examined. Accordingly, 
the difference between the total scores of the post-test and pre-test of the experimental and control groups was 
found to be significant (F/, 5.77, p<.05). According to this finding, it could be said that the support of the envis- 
aged program's activities with AR applications had increased students’ success and reflected a moderate effect 
(n?=.09). Finally, there was a significant difference between SSBST post-test and pre-test scores of all students in 
experimental (1) and control (1) groups (without group distinction) (Fiza, =202.85; p<.05). It could be stated that 
no matter which activities are applied, the students’ success significantly increased throughout the experimental 


process, from the pre-implementation to the post-implementation, and this increase reflected a broad effect 
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(n?=.78). Then, the arithmetic average and standard deviation values for SMSL pre-test, post-test and difference 
scores of Experimental (1) and Control (1) groups were examined (Table 12). 


Table 12. The arithmetic mean and standard deviation values for SMSL pre-test, post-test, and difference 
scores of experimental (1) and control (1) groups. 


Pre-Test Post-Test Difference Score 
Scale Group N 
XxX SD xX SD XxX SD 
Experimental (1) 30 130.63 17.33 136.30 13.94 5.67 14.72 
SMSL 
Control (1) 30 121.70 18.94 122.77 16.53 1.01 21.69 


As shown in Table 12, the SMSL scale score of Experimental group (1) was Xoreter=! 30.63 before the applica- 
tion, whereas X vost tet! 36.30 after the experimental process. The success score of the Control group (1) was yd 
test | 21.70, while the post- test was X voseten= | 22.77.|n addition, while the difference between post-test and pre-test 
scores of experimental group (1) was Keser Snag oO while the difference between post-test and pre-test scores 
of control (1) group was Gn. Coate .01. Two-way ANOVA was then conducted for mixed measurements to 


determine students’ motivation for science learning and is given in Table 13. 


Table 13. Results of two-way analysis of variance for mixed measurements for SMSL pre-test and post-test 
scores of experimental (1) and control (1) groups. 


Source of Variance Sum of Square df Le 7 F p n2 
quare 

Between-Subjects Factors 25097.30 59 
Experimental procedure 3785.63 1 3785.63 10.30 .0001 15 
Within-Subjects Factors 11852.00 60 
pire, Dewean Pe lest 340.03 1 340.03 174 19" 03 
and post-test 
pilerence between pre-test and post- 158.70 1 158.70 81 37 or 
test “Experimental procedure 


Total 36949.30 119 


According to Table 13, it can be seen that the SMSL scores of the experimental groups which were taught with 
the AR applications and the control groups which were taught according to the activities in the envisaged program 
did not show any significant difference after the experimental intervention (F,, 5-81 ; p>.05). When the difference 
scores of the pre-test groups (Experimental group (1), Kissa Boe OU Control group (1), DFO TR | .01) 
are taken into account; there appears to be less motivation score changes in both groups after the application. 
The fact that SMSL was already high before the application, the student group was accustomed to the researcher, 
and it was very difficult to change the affective variables such as motivation within a period of six weeks, may be 
effective on having no significant difference. Although there was no significant difference, the increase in the dif- 
ference scores in the experimental group shows that the AR applications are influential on the motivation of the 
students. The difference between the sum of the post-test and pre-test SMSL scores of the experimental and control 
groups was found to be significant (F/, ..._10.30; p<.05). According to this finding, it can be said that supporting the 
envisaged program's activities with AR applications increased the SMSL and reflects a wide effect (n?=.15). Finally, 
there was no significant difference between the SMSL post-test scores and pre-test scores of all the students in the 
Experimental (1) and Control (1) groups (without group distinction) (Fisay=! .74; p>.05). This is due to the fact that 
the difference between the sum of the post-test scores and the pre-test score of the groups which received the 
pre-test was very small. As noted above, it is difficult to change affective variables, such as motivation, in a short 
period of time, and the fact that students are already motivated in science lessons, even if they are not subjected 


to any treatment, these factors can be shown to cause no significant difference. 


372 


URS https://doi.org/10.33225/jbse/19.18.362 


Journal of Baltic Science Education, Vol. 18, No. 3, 2019 


ISSN 1648-3898 /Print/ THE EFFECT OF THE USE OF AUGMENTED REALITY APPLICATIONS ON THE AGADEMIG 
SUCCESS AND MOTIVATION OF 7TH GRADE STUDENTS 
ISSN 2538-7 138 /onines (P. 362-378) 


Results Related to the Third Question 


The third question of the study was declared as “When pre-test is controlled, does the experimental proce- 
dure make a significant difference between the academic success and SMSL post-test scores of the students in 
the pre-test groups?” That is to say, it is intended to determine whether the experimental process is effective or 
not by controlling the pre-test applied to Experimental (1) and Control (1) groups. For this purpose, “Single Factor 
Covariance Analysis (ANCOVA)” was conducted. In the analysis of covariance, the effect on the dependent variable, 
namely pre-test, was determined as the variable to be controlled. Since the pre-test was checked on this count, the 
actual effect of the experimental process was tried to be found. For the covariance analysis, pre-test and post-test 
success scores of Experimental (1) and Control (1) groups showed homogenous and normal distribution and it 
was identified that there was a linear relationship between pre-test success scores (control variable) and post-test 
success scores (dependent variable) and that the slopes of the regression lines were equal. The post-test scores 
corrected according to the pre-test scores of the Experimental (1) and Control (1) groups were given in Table 14. 


Table 14. Corrected mean scores calculated by considering the SSBST pre-tests of experimental (1) and control 


(1) groups. 
Group N Mean Corrected Mean 
Experimental (1) 30 84.33 83.43 
Control (1) 30 70.33 70.97 


According to Table 14, when the effect of the pre-test was controlled, the average success score of the Ex- 
perimental group (1) decreased from 84.33 to 83.43, and the success score of the control group (1) increased from 
70.33 to 70.97. The results of the covariance analysis (ANCOVA) conducted to understand whether the differences 
between the adjusted average scores of the groups were meaningful, were given in Table 15. 


Table 15. One-factor covariance analysis (ANCOVA) of the corrected post-test scores of SSBST of the experi- 
mental (1) and control (1) groups. 


Source of Variance Sum of Square df Mean of Square F p n2 
Pre-test 3477.16 1 3477.16 16.26 .0001* 22 
Experimental procedure 2293.77 1 2293.77 10.72 .0001* 16 
Total 17964.31 59 

*9<.05 


According to the results of the covariance analysis (ANCOVA) in Table 15, when the SSBST pre-test scores were 
kept constant, it was found that there was a significant difference between the post-test scores of Experimental (1) 
and Control (1) groups (F,, .,=10.72; p<.05). These results showed that teaching the lecture with AR applications 
reflects a broad effect in increasing students’ success (n?=.16). In addition, when the effect of the control variable 
is examined in order to control the experimental process, it was seen that, there is a significant relation between 
pre-test and post-test (F/, ,,=16.26; p<.05). Then for covariance analysis, it was determined that pre-test and post- 
test SMSL scores of Experimental (1) and Control (1) groups showed homogeneous and normal distributions; it 
was determined that there was a linear relationship between pre-test SMSL scores and post-test SMSL scores, and 
the inclination of regression lines was equal. It has been identified. The post-test scores, which were corrected ac- 


cording to pre-test scores of Experimental (1) and Control (1) groups, were given in Table 16. 
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Table 16. Adjusted mean scores calculated by considering the pre-tests of SMSL of experimental (1) and control 


(1) groups. 
Group N Mean Corrected Mean 
Experimental (1) 30 136.47 135.14 
Control (1) 30 122.77 123.93 


According to Table 13, when the effect of pre-test was controlled, the SMSL average of Experimental group 
(1) decreased from 136.47 to 135.14, and the SMSL average of Control group (1) increased from 122.77 to 123.93. 
The results of the covariance analysis (ANCOVA) on whether the difference between the adjusted average scores 
of the groups are meaningful, were given in Table 17. 


Table 17. One-factor covariance analysis (ANCOVA) of SMSL scale on corrected post-test scores of experimental 
(1) and control (1) groups. 


Source of Variance Sum of Square df Mean of Square F p n2 
Pre-test 1297.27 1 1297.27 6.03 .02* 10 
Experimental procedure 1772.55 1 1772.55 8.24 .01* 13 
Total 15325.22 59 

*P<.05 


According to the covariance analysis (ANCOVA) result in Table 17, when the SMSL pre-test scores were kept 
constant, it was found that there was a significant difference between the post-test scores of Experimental (1) and 
Control (1) groups (F., .,.=8.24; p<.05).This suggests that teaching the course with AR applications reflects a mod- 
erate effect in increasing the motivation of students (n?=.13). In addition, when the effect of the control variable 
was examined in order to control the experimental process, it was seen that, there is a significant relation between 
pre-test and post-test (F, .. =6.03; p<.05). 


(1,57) 


Discussion 


Solomon four-group model enabled to determine whether pre-testing, conducted prior to the start of 
the study, affects the academic success and motivation levels of students. Analysis results indicate that pre- 
testing did not have a significant effect on the academic success and motivation levels of the study groups. 
Similar results have been reported previously in the literature. Kartal (2013) has conducted a mixed study to 
examine the impact of microteaching on the development of pedagogical subject matter knowledge of science 
teacher candidates. Quantitatively, the study was based on Solomon four-group model and it was found that 
pre-testing did not influence the results significantly. Dogan (2017) conducted an experimental study using the 
Solomon four-group model to investigate the effect of teaching the subject “earthquake” via digital games in 
the social studies course on the academic success. The results indicated that there was a significant difference 
between the scores of experimental and control groups and pre-testing did not have a significant influence. 
All et al. (2017) conducted a study using the Solomon method on the evaluation of game-based learning and 
aimed to determine the advantages and disadvantages of pre-testing. As a result, it was determined that the 
pre-test had no effect and the game-based learning was effective. However, they emphasized that the means 
of pre-test groups were higher than those without pre-test and this finding should be taken into consideration 
in other studies. 

Although there is no significant difference between the study groups in terms of pre-test, it is observed 
that the scores of the groups who are stimulated in terms of pre-test were higher when the final test scores of 
the students were taken into consideration. This can be explained by the fact that students are familiar with 
the structure and form of the questions. Previous studies conducted in our country using Solomon's Four 
Group Model were examined and it was observed that there was no clear information about the statistical 
operations to be used in the analysis of the data of this experimental pattern. For this reason, the studies from 
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abroad, describing Solomon’s Four Group Models, were taken into consideration. In this study, data obtained 
by using Solomon's Four-Group Model was analyzed rigorously and effort was made to transmit each step 
clearly to the reader. Solomon's four-group model, which allows the determination of whether the effect of 
pre-test sensitivity is meaningful, has strengthened the internal validity and external validity of the study. The 
lack of sensitivity to pre-test indicates that the changes in academic success and motivation of students are 
due to the AR technology. 

According to the two-way analysis of variance to determine the effect of experimental process, it has 
been observed that the use of AR applications in the course process has a positive effect on students’ academic 
success. One of the reasons for this positive effect is that the students encounter a new technology, find this 
technology interesting, and perceive it as a magic, due to AR technology and objects appearing on the paper 
in three dimensions. Previous studies also reported similar results that show that the use of AR increases the 
academic success of students. Sirakaya et al. (2018) reported that the use of AR applications in the seventh 
grade science class “Solar System” unit increased the academic success of students. Ozarslan (2013) conducted a 
quasi-experimental study and found that the AR applications have a positive influence on the academic success 
of students. Baysan (2015) also reported that AR-supported learning environments increase the academic 
success of students. Klick (2015) investigated the effect of mobile AR applications on the academic success 
and cognitive burden of medical students and found that the experimental group was more successful than the 
control group. Chiang et al. (2014) reported that mobile-supported AR applications increase academic success 
and motivation levels. Bacca et al. (2014) indicated that the AR is effective in improving the academic success. 
Chien et al. (2010) found that AR training strengthens spatial memory by providing an interactive interface that 
provides better anatomy learning. Maier et al. (2009) developed an AR program called “Augmented Chemical 
Reactions” to teach chemical reactions in chemistry lessons, which facilitated learning of the underlying 
processes and reduced the fear of chemistry. 

The transfer of knowledge learned in the field of education in the direction of these studies is possible 
with the interactive environment presented to the students by the AR applications. As a matter of fact, new 
technologies integrated with education attract attention of students, activate them in the learning process, 
increase their occupation and motivation, and help them to understand the subject more easily. The fact 
that the world and the universe learning domain will be handled as the first units in the new 2017 Science 
Education Program shows that this unit is now given a higher emphasis. Previously, this unit was not given the 
attention it deserved due to it being the last unit and upcoming summer holiday. There is also a problem with 
the implementation of the activities envisaged in the program concerned with this unit. This is because there 
is no learning environment in which students can observe celestial objects. Textbooks are also insufficient in 
terms of visualizing the solar system in three dimensions. The teaching of astronomy, which is usually carried 
out in closed environments, is perhaps of interest to the students but obviously not very efficient. In this 
study, three-dimensional visualization of astronomical phenomena that cannot be applied in a typical class 
environment under normal conditions was achieved by the use of AR applications and thereby easier learning 
was provided by establishing the spatial link between these concepts in accordance with the cognitive period 
of seventh grade students. 

When the effect of the experimental procedure on the motivation of the students is examined, it has been 
observed that instruction with AR applications affects the motivation of students at a significant level. There is 
also research in the literature showing that AR-enhanced learning increases the motivation of students and they 
enjoy using these applications. When the studies carried out in this context are examined, Tan and Lui (2013) 
found that using AR technology in teaching English vocabulary influenced the motivation of students positively. 
Di Serio et al. (2013) has stated that the attention and motivation of the students increased significantly and 
increase in their interaction with the learning environments has increased the motivation of the students when 
AR was used in the visual arts lesson. Similarly, Mahadzir and Phung (2013) found that the use of AR technology 
in books in foreign language teaching increased student motivation in vocabulary learning and Delello (2014) 
stated that when prospective science teachers used AR technology in the science class, they stated that it hada 
positive impact on the learning environment, increased students’ motivation and provided ease of application. 

It is also important to mention some limitations of the AR technologies used in the research. These 
limitations usually occur in the sense of follow-up, hardware, connection and improvement works are being 
done to overcome these limitations. Especially, late-detecting of the object on the application by camera, 
freezing of the camera and, the most importantly, excessive power consumption while the application is open 
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are the serious disadvantages. In order for AR technology to work properly, these technical problems need 
to be addressed and for this purpose, work-force and time are needed. Although AR faces some problems, it 
is possible to close this gap with the rapid development of digital and mobile technologies in recent years. 
Another limitation observed in the study was that in order for the applications to operate in a healthy manner 
and for each student to observe the objects in three dimensions, the study period in the experimental groups 
should be longer than in the control groups. The probable reason, why students in the experimental group 
finished their experience longer than others, could be that those students wish to observe the solar system 
in three-dimension more than once, as well as the time needed for the technological tools to operate and for 
the object recognition of the AR applications. 

It is possible to say that mobile devices such as mobile phones and tablets are suitable environments for 
today’s technology for AR experience. Mobile devices have advantages such as being portable, being widely 
used, being easily recharged and including software projects as Play-Store and Apple-Store. When the progress 
on mobile devices from past to present is examined, it is not wrong to predict that higher-capacity mobile devices 
will be produced in near future. Indeed, this indicates that new AR applications will likely to be designed in 
future. As such, mobile AR applications will become important material for educators. In this context, educators 
can make innovative choices in the classroom, including AR applications in addition to tools such as textbooks, 
board and chalk. Particularly, the AR technology will help students with weaker visual perception with their 
three-dimensional thinking skills. Thanks to the AR technology, real images of virtual objects are constructed, 
and stationary objects are transformed into multimedia, increasing the functionality of the learning environment. 
The increasing diversity of educational technologies gives educators important duties and roles in recognizing 
and using them effectively. For this reason, educators should follow the technological developments and try 
to use the most appropriate tools for their fields (Akkoyunlu, 2002). The body of literature in this field shows an 
increasing awareness of teachers, in particular towards the usage of AR technologies in education as well as an 
increasing number of research studies. Taking this study and similar studies as references, further studies on 
different topics and courses will be important to demonstrate the effectiveness of this technology in education. 

Considering the compatibility of the use of mobile AR applications in teaching of other subjects in the 
science course, this technology may as well be used for the teaching of other units that are similarly difficult 
to visualize. The physical environments of schools should be designed in such a way that they can be utilized 
in the best possible way to benefit from AR applications. Classrooms should be donated with the basic tools 
and equipment required for this technology. In addition, studies using different methods and variables can 
be done with a larger sample group on the effective usability of this technology. Considering that developed 
countries aim to train individuals who are able to adapt technology rapidly, use technology efficiently and, most 
importantly, produce technology while creating their education programs (Can & Kaymakci, 2017), the successful 
integration of the AR technology into educational environments and the assessment of its advantages and 
disadvantages with these studies will be useful to give an idea to educators who want to use this technology. 
Finally, from the methodological point of view, it is suggested that further research is conducted to observe 
the effect of pre-test and experimental procedure due to the lack of studies conducted on the use of Solomon 
four-group model in science education in Turkey. 


Conclusions 


Thanks to the developments in mobile devices, the AR technology has been used in many sectors such 
as health, marketing, education, cinema, advertising and design. Thus, it has become possible to bring AR 
applications into the class and laboratory as teaching tools on mobile devices. In this study, AR applications 
are thought to be effective tools to provide access and visualization of concepts that are difficult to learn 
in astronomy. In this way, students will be able to discover the unknown properties of the universe and the 
inaccessible places of it. Thus, students will have the opportunity to experience and learn the universe in a 
realistic and better way. These experiences are expected to have a positive effect on their academic success 
and increase their motivation. 

The general conclusion of this research is that using augmented reality applications in science teaching 
significantly contributes to the improvement of students’ achievement and motivation. Since this research was 
conducted within the topic of Solar System and Beyond, the results cannot be generalized to the entire science 
teaching. Of course, the use of AG technology is not the best possible solution. For future research, it will be 
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useful to support science lessons with AG technology in order to have a significant effect on students’ science 
achievement. This research was conducted by using “Solomon Four-Groups Design’, which is one of the most 
powerful models that ensure internal and external validity at the same time. The Solomon four-groups design 
method enabled us to obtain more valid and reliable results since the effect of pre-test and intervention could 
be examined in detail. In the future, with the reporting of the successful integration of AG applications into 
educational environments and the obvious advantages on the educational environment, it is expected to give 
an idea and courage to educators who want to use this technology. 
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